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EXECUTIVE  SUMMARY 


One  of  the  tasks  proposed  in  the  research  effort  from  which  this 
report  has  stemmed  was  to  "...explore  the  feasibility  of  developing  a 
computerized  diagnostic  report  generator."  As  the  body  of  the  report 
Indicates,  we  have  been  jble  to  go  considerably  beyond  simple  feasi- 
bility exploration.  Presented  here  is  a description  of  CANOPUS 
(a  computerized  diagnostic  and  prescription  software  system),  together 
with  an  account  of  progress  made  to  date  in  its  construction.  We 
anticipate  that,  when  complete.  It  will  form  a useful,  serviceable  tool 
for  HRM  consultants  and  their  client  units. 

STATEMENT  OF  THE  PROBLEM 

An  organization  is  an  open  social  system,  which  means  that  it 
functions  by  receiving  inputs  of  resources  and  energy  from  the  outside 
world,  converts  them  by  a throughput  process  to  a commodity  or  service 
which  It  then  exports  into  the  environment  In  return  for  replenishment 
of  its  resource  input.  In  greatly  oversimplified  form,  one  might  view 
the  Navy  in  social  systems  terms  as  receiving  Inputs  from  American 
society  in  the  form  of  manpower  from  the  civilian  population  and  money 
aporoprlated  by  Its  Congress.  The  Navy  by  Its  functioning  converts 
these  resources  Into  an  output  of  defense  of  the  nation,  which  it 
"exports,"  in  the  sense  that  It  makes  it  visible,  present,  and  useful 
in  the  world. 
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In  the  Navy,  as  in  any  system,  not  all  of  the  input  appears  at  the 
end  of  the  cycle  in  the  form  of  output.  Some  of  the  input  must  necessarily 
be  consumed  in  the  throughput  process  itself;  that  is,  some  proportion 
must  be  diverted  to  maintain  the  organization.  The  more  of  the  input 
that  must  be  so  diverted,  in  relation  to  a given  output,  the  less  effec- 
tive the  organization  is. 

The  efficiency  of  the  throughput  process  therefore  largely  determines 
the  organization's  effectiveness,  and  it  is  to  the  Improvement  of  this 
ratio  that  organizational  development  activities  such  as  the  Navy's 
Human  Resources  Management  effort  are  directed. 

In  general,  development  (in  the  view  of  those  who  study  living 
systems)  is  a function  of  the  appropriate  contlnatior  of  two  kinds  of 
things:  (1)  characteristics  of  the  system  which  predispose  it  to  experience 

certain  kinds  of  change  (such  as  particular  leadership  practices)  and 
(2)  events  external  to  it  which  cause  or  "trigger"  the  change  to  otcur 
when  It  does  (such  as  training  or  Intervention  activities).  Character- 
istics such  as  leadership  practices  exist  before  the  consultant  arrives 
on  the  scene  and  are  necessarily  quite  varied  in  form  and  style.  Any 
specific  thing  that  the  consultant  does  by  way  of  training  or  intervention 
is  unlikely  to  be  appropriate  to  all  (or  even  any  large  number)  of  such 
situations.  To  affect  the  functioning  of  the  unit  constructively,  there- 
fore, requires  that  one  carefully  match  the  intervention  activity  to  the 
ne.:ds  of  each  client  unit.  This,  then,  is  the  problem  of  diagnosis  in 
organizational  development--doscr1bing  organizational  characteristics  in 
sufficient  detail  and  with  enough  accuracy  to  permit  one  to  match 
available  intervention  activities  to  diagnosed  conditions  in  ways  likely 
to  Improve  effectiveness. 
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THE  PROPOSED  PROCEDURE 

It  is  to  this  set  of  issues  that  CANOPUS  is  addressed.  As  a procedure, 
its  unique  features  are  the  following: 

(1)  It  is  almost  entirely  computerized,  which  permits  it  to 
take  account  of  a wider  array  of  information  with  greater 
reliability,  validity,  and  speed  than  would  be  possible 
by  methods  which  rely  upon  more  "clinical"  procedures. 

(2)  It  prioritizes  problems  for  attention  by  the  manager  or 
consultant,  not  only  in  terms  of  the  level  of  goodness 
or  badness  of  the  characteristics,  but  in  terms  of  their 
relationships  to  performance  criteria  as  well. 

(3)  It  goes  beyond  the  simple  description  oi  strengths  and 
problems  to  an  assessment  of  their  causes  (situational 
factors,  information  needs,  skill  deficiencies,  and 
values  conflicts). 

(4)  It  recomnends  possible  training  or  intervention  steps 
which  accumulated  experience  suggests  are  optimal  for 
situations  like  the  one  in  hand. 

(5)  It  summarizes  both  conditions  and  treatment  steps  across 
organizational  entitles  and  generates,  by  computerized 
text-writing,  a written  report  for  managers  and  their 
consultants . 

Not  all  of  the  steps  are  at  present  constructed.  Two  of  the  initial 
and  most  difficult  are,  however,  and  it  is  our  hope  that  the  remaining 
ones  will  be  completed  within  the  next  few  months. 
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CONTENTS  OF  THE  REPORT 

From  this  brief  overview,  the  reader  may  turn,  if  he  chooses,  to 
the  body  of  the  report.  The  first  section  contains  a review  of  the 
problems  and  issues  present  in  organizational  diagnosis.  We  anticipate 
that  this  section  will  be  especially  useful  to  consultants,  who  face 
almost  dally  the  problems  involved  in  diagnosis  and  acdcn  planning. 

The  second  section  describes  in  some  detail  the  CANOPUS  procedure  and 
its  components,  while  the  appendices  provide  documentation  of  the  first 
two  such  components.  These  sections,  we  feel,  will  be  of  special 
interest  to  those  who  must  handle  and  analyze  survey  data  for  develop- 
ment purposes. 

A CONCLUDING  NOTE 

In  designing  and  beginning  the  construction  of  CANOPUS,  it  has  been 
our  intention  to  provide  a tool,  not  a stralghtjacket.  Like  any  tool 
which  depends  upon  systematically  stored  experience,  this  one  should 
grow  In  strength  and  usefulness  as  It  is  used.  It  Is  our  hope,  there- 
fore, that  it  will  be  not  only  useful,  but  used. 
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ORGANIZATIONAL  DIAGNOSIS:  CONCEPTS,  ISSUES,  AND  METHODS 

< 

The  research  proposal  stated  the  following: 

An  Indicator-based  development  program  such  as  the  Navy  j 

Human  Goals  Plan  is  only  as  good  as  the  Information  system  upon  1 

which  it  is  based.  Data  bank  storage,  retrieval,  analysis,  and 
processing  capabilities  (simlla, , in  at  least  some  aspects  to 
those  already  In  place  in  the  civilian  sector  in  our  own  data 
3 bank)  must  be  established,  tested,  and  subjected  to  at  least  a 

preliminary  evaluation. . .as  part  of  this  we  would  explore  the  1 

feasibility  of  developing  a computerized  diagnostic  report  /' 

generator,  (p.3) 

. ! 

The  purpose  of  this  present  technical  report  Is  to  fulfill  the  intent  i 

of  that  concluding  phrase.  As  the  reader  will  soon  see,  we  have  been  able 
to  go  considerably  beyond  the  slnple  feasibility  exploration  proposed  at 
the  outset.  A computer  software  package  has  been  developed  which  provides  j 

a basic  capability  of  computerized  diagnostic  report  generation  most 
t particularly  geared  to  the  organizational  development  process. 

In  the  sections  which  follow,  we  shall  review  first  of  all  the  issues, 
methods,  and  problems  of  diagnosis  in  organizational  development,  and  then 
% in  later  sections,  describe  the  form  and  substance  of  the  procedure  which 

has  been  developed  in  the  present  instance.  Documentation  of  the  conputer  j 

software  Is  attached  as  Appendix  A followed  by  sample  printout  produced  by  j 

that  software  as  Appendix  B.  J 

i 

The  Problem  of  Diagnosis  in  Organizational  Development  I 

An  organization  is  an  open  social  system.  In  its  most  rudimentary 
form  this  means  that  it  functions  by  receiving  inputs  of  resources  and 
energy  from  the  outside  world,  converts  them  by  a throughput  process  to  a 

commodity  or  a service  which  It  then  exports  Into  the  environment  in 

« 

I 

1 
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return  for  the  replenishment  of  Its  resource  input.  Thus,  the  cycle 
repeats  and  continues.  The  process  is  illustrated  in  Figure  1 both  in 
its  general  form  of  an  input-throughput-output  sequence,  and  more  specifi- 
cally as  it  applies  to  the  Navy.  In  greatly  oversimplified  form,  one 
might  view  the  Navy  in  social  system  terms  as  receiving  inputs  from 
American  society  In  the  form  of  manpower  from  the  civilian  population,  and 
money  appropriated  by  its  Congress.  The  Navy  by  its  functioning  converts 
these  resources,  the  energy  and  talents  of  the  people,  and  the  financial 
resources  appropriated  by  legislation  into  an  output  of  defense  of  the 
nation,  which  it  ’'exports,"  In  the  sense  ti.  t it  makes  it  visible,  present, 
and  useful  in  the  world.  In  return  for  that  output,  more  manpower  accrues, 
new  financial  resources  are  appropriated,  and  thus  the  cycle  repeats 
itself.  In  abstract  terms  the  Navy's  effectiveness  as  an  organization  is 
represented  by  the  amount  of  such  national  defense  output  which  it  is  able 
to  generate  for  the  inputs  received. 

In  any  system,  not  all  of  the  input  appears  at  the  end  of  the  cycle 
in  the  form  of  output.  Some  of  the  input  must  necessarily  be  consumed  in 
the  throughput  process  itself.  Organizations  live  and  exist,  and  must  be 
maintained.  A portion  of  the  input  in  any  system,  therefore,  is  diverted 
to  the  maintenance  of  its  internal  functioning.  The  more  of  the  input 
that  must  be  so  diverted,  In  relation  to  a given  output,  the  less  effective 
the  organization  is.  Stated  in  more  dynamic  terms,  an  Increase  In  effec- 
tiveness amounts  to  attaining  more  and/or  better  output  for  the  same  input, 
or  of  attaining  the  same  output  with  less  input. 

As  the  only  remaining  variable,  the  efficiency  of  the  throughput 
process  Itself  largely  determines  this  characteristic.  It  is  to  this 
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tiveness amounts  to  attaining  more  and/or  better  output  for  the  same  Input, 
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process  Itself  largely  determines  this  characteristic.  It  is  to  this 
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problem  that  organizational  development  addresses  itself,  since  it 
represents  a systematic  planned  attempt  to  upgrade  aspects  of  the  through- 
put process  These  upgrading  attempts  are  basically  Instances  of  planned, 
purposeful,  constructive  change.  Change,  in  the  view  of  those  who  study 
living  systems  is  In  turn  a function  of  the  appropriate  conbination  of  two 
kinds  of  things:  (1)  characteristics  of  the  system  Itself  which  predispose 

it  to  experience  certain  kinds  of  change,  and  (2)  events  external  to  it 
which  cause  or  "trigger"  the  change  to  occur  when  it  does.  A simple  illus- 
tration may  suffice-- jet  fuel  is  conbustible,  that  is,  it  has  a propensity 
to  bum.  For  a fire  to  occur  aboard  ship  Involving  aviation  fuel  requires 
a conbination  of  that  fuel  itself  and  an  appropriate  trigger,  for  example, 
a flaming  match.  A flaming  match,  or  a lit  cigarette,  in  a pool  of  loose 
fuel  will  trigger  a fire.  A biscuit,  or  a handkerchief,  dropped  in  that 
same  pool  of  fuel,  will  not  trigger  combustion. 

Simple  and  perhaps  absurd  as  this  illustration  Is,  it  describes  in 
many  ways  the  issue  lying  at  the  heart  of  organizational  development. 
Organizations  have  characteristics  which  coi»|>rise  their  functioning,  for 
exanple,  the  climate  present  In  the  organization,  the  supervisory  behavior 
of  its  managers,  the  mutual  behavior  of  subordinates  in  the  work  setting, 
and  the  processes  of  the  groups  that  make  it  up.  For  any  particular 
subordinate  unit,  its  particular  combination  of  bahavlors  and  conditions 
predispose  it  toward  being  affected  by  certain  external  conditions,  not 
by  others.  Organisational  development  interventions,  training,  and  even 
changes  in  the  external  world  In  which  It  lives,  vary  In  the  impact  which 
they  have  on  different  conbinations  of  such  conditions.  To  affect  the 
throughput  process  of  an  organizational  unit  constructively,  what  one 
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must  do  is  be  certain  that  an  appropriate  set  of  Interventions  inpinge 
upon  the  conditions  in  that  unit.  Necessarily,  ro  intervention,  training 
activity,  or  external  event.  Is  relevant  to  all  situations.  Rather  by 
their  very  nature,  particular  interventions  selectively  influence  organi- 
zations with  certain  characteristics  rather  than  others.  Matching  the 
intervention  to  the  organization  and  its  configuration  of  behaviors, 
strengths,  and  problems,  requires  that  one  understand  what  that  configura- 
tion of  strengths  and  problems  is  This  is  in  effect,  a problem  of  diag- 
nosis in  organizational  development--describing  organizational  character- 
istics in  sufficient  detail,  and  with  enough  accuracy  to  permit  one  to 
match  available  intervention  activities  in  ways  likely  to  profice 
constructive  change. 

As  a discernible  fir  Id,  organizational  development  is  conparati vely 
young.  Burke  (1973)  has  indicated  that  It  Is  barely  15  years  old,  although 
various  elements  that  currently  are  viewed  as  comprising  it  date  from  an 
earlier  period.  As  a whole— let  alone  an  integrated  whole--1t  seems 
indisputable  that  the  field  is  still  in  its  infancy.  As  a field,  it  traces 
its  lineage  in  two  related,  but  different,  directions:  (1)  the  world  of 

the  practitioner,  as,  for  example,  adult  education,  clinical  practice,  and 
.the'TIke,  and  (2)  the  utilization  of  scientific  research  findings  in 
organizational  behavior.  From  the  former  has  sterrmed  an  action  orienta- 
tion, together  with  a growing  concern  for  accomplishing  action  objectives 
more  expeditiously  and  in  higher  quality  fashion.  From  the  latter 
(scientific  research),  the  field  has  inherited  from  the  cannons  of  science  a 

concern  for  accuracy,  along  with  an  awareness  of  the  importance  of 
implementing  research  findings. 
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Melded  and  interwoven,  these  dual  sets  of  concerns  have  led  to  the  general 

recognition  that  at  least  two  major  processes  (not  just  one)  are  involved  in 

the  0.0.  field--diagnosis  and  treatment  (Bowers  & Norman,  1969).  The 

result  is  that  diagnosis  has  received  increasing  attention  in  recent  years. 

Prominent  writers  in  the  field  have  talked  at  moderate  length  about  the 

necessity  for  a solid  diagnosis  as  a basis  for  action.  Practitioners, 

similarly,  have  talked  repeatedly  about  the  role  played  in  their  efforts 

by  a process  that  they  see  as  diagnostic.  Despite  this  increased  attention, 

much  would  appear  to  constitute  less  than  a satisfactory  approach  to  the 

problem.  While  Increasingly  talked  about,  diagnosis  would  appear  more  often 

to  be  paid  lip-service  than  to  receive  serious  attention.  The  following 

quotation  from  Levinson  Illustrates  the  nature  of  the  problem: 

That  brings  us  to  the  problem  of  diagnosis...!  do  not  yet 
see  that  ki*1  of  diagnostic  process  in  the  literature  or  in 
practice  which  leads  to  intervention  of  choice:  given  an 
organization  of  a given  ki.id,  at  a given  point  in  Its  develop- 
ment, with  given  kinds  of  constituencies  and  groups  of 
employees,  with  a given  kind  of  leadership  and  a given  set  of 
problems,  what  should  the  strategy  of  Intervention  be?  toward 
vjhat  anticipated  outcome?  (Levinson,  1973,  p.  201) 

To  serve  the  purposes  which  Its  name,  and  Its  role.  Intends,  a diag- 
nosis should  constitute  an  analysis  of  the  current  functional  state  of  a 
particular  system  for  purposes  of  determining  appropriate  treatments 
(action  steps,  or  Interventions).  Both  the  name  and  this  brief  definition 
Imply  a nunfcer  of  characteristics  which  should  be  reflected  In  any 
methodologically  sound  diagnostic  effort: 

(1)  Comp re hen si  vents s - since  problems  may  originate  and  occur  in 
any  part  or  aspect  of  a system,  any  diagnostic  effort  worthy  of 
the  name  should  treat  comprehensively  the  properties  of  the  system 
as  such,  that  is,  it  should  not  look  selectively  at  a few  aspects 
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of  system  functioning,  ignoring  the  rest.  It  should,  instead, 
make  as  complete  an  effort  as  possible  at  assessing  the  total 
functional  state  of  the  system. 

(2)  Theoretical  Anchorage  - a good  diagnosis  should  look,  not  merely 
at  an  arbitrary  array  of  properties,  but  at  an  array  of  constructs 
which  reflect  an  underlying  scheme  that  is  Itself  sensible,  and 
which  has  been  derived  from  the  real  world  by  a process  of  solid 
research.  This  framework  serves  to  provide  explanatory  power  by 
indicating  how  in  general  (that  is,  in  most  such  organizations) 
various  aspects  of  functioning  should  relate  to  one  another. 
Without  this,  the  problem  becomes  an  assessment  based  upon  a 
somewhat  haphazard  collection  of  readings. 

(3)  Precedence  - both  its  name  and  its  role  imply  that  diagnosis 

precede  and,  in  part  at  least,  determine  which  particular  treat- 
ment from  an  array  of  possible  treatments  should  be  used  in  the 
situation  at  hand. 

(4)  Orientation  - diagnosis  Implies  an  orientation  on  the  part  of 
the  consultant  primarily  toward  the  client  system's  well-being 
rather  than  simply  toward  his  own. 

(5)  Differential  - diagnosis  Implies  a differential,  that  Is,  that 
there  are  different  states  of  nature  which  the  employed  assess- 
ment techniques  distinguish  from  one  another,  and  which 
ultimately  have  different  action  implications. 

By  way  of  contrast,  there  are  several  things  that  a diagnosis  should 
not  be.  It  should  not  be  a simple  benchmark  against  which  to  measure 
progress.  Used  in  this  way,  diagnosis  would  amount  to  little  more  than 
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an  evaluation,  since  It  would  carry  the  Implication  that  treatment  is 

i 

determined  on  some  basis  other  than  system  assessment.  Despite  this  fact,  \ 

# 

there  would  appear  to  be,  at  least,  some  Instances  in  which  practitioners  I 

i ! 

welcome  measurement,  not  for  the  treatment  guidance  which  it  provides,  but 
simply  tr  provide  possibilities  for  "proof  positive."  j 

Similarly,  a diagnosis  should  not  be  merely  a "map  of  pitfalls"  which 
permit  the  consultant  to  do  what  he  always  does  anyway,  but  with  minimum  1 

risk  to  Mmself  and  others.  Using  it  as  a map  of  pitfalls  means  that  its 
role  ir  determining  treatment  is  denied  It.  This  use  biases  as  well,  to 
some  extent  at  least,  against  an  orientation  toward  the  client  system's 
well-being  first  and  foremost,  and  even  perhaps  negates  a bit  the  differ-  | 

ential  character  which  Is  so  necessary  for  effective  guidance  of  the 
treatment  process. 

Finally,  differential  diagnosis  is  not  simply  a matter  of  variety 

among  consultants.  Occasionally  writers  and  practitioners  cite  the  variety 

of  things  done  by  different  consultants,  or  change  agents,  as  evidence  for 

the  eclecticism  present  In  the  field,  and  Infer  from  this  that  treatments 

are  indeed  differentially  selected  on  a diagnostic  basis.  Levinson,  In 

the  article  previously  cited,  correctly  describes  the  misleading  nature  of 

such  a representation.  The  fact  that  different  consultants  employ 

different  techniques  says  nothing  about  the  extent  to  which  any  one  of 

them  selects,  from  among  a wide  array  of  quite  different  treatments,  the 

one  which  he  will  in  fact  use  In  a specific  Instance  upon  the  basis  of  a 

solid  diagnosis  of  that  system's  own  functioning.  The  eclecticism.  In 

other  words,  Is  not  a matter  of  simple  variety  In  methods  and  practices 

among  consultants,  each  of  whom  may  well  be  highly  consistent  In  what  he 

does  from  one  situation  to  the  next.  < 
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We  thus  arrive  at  a view  that  solid,  rigorous  diagnosis,  differ- 
ential in  character,  comprehensive  against  a theoretical  framework,  is  an 
essential  step  in  determining  which  treatment  to  use  for  purposes  of 
enhancing  a prrtlcular  client  system's  well-being.  It  is  not  a simple 
benchmark;  It  Is  not  merely  a map  of  pit-falls;  nor  is  it  a matter  simply 
of  pointing  to  the  practice  differences  among  consultants.  Its  usefulness 
depends  upon  the  care  which  has  been  used  In  doing  It,  and  It  is  to  this 
Issue  that  we  now  turn  our  attention. 

Collecting  Diagnostic  Data 

Although  an  infinite  nunber  of  data  collection  methods  may  be  poss- 
ible, generlcally  they  would  appear  to  fall  Into  a quite  limited  number  of 
categories.  Each  has  its  strengths  and  Its  weaknesses,  and  across  the  lot 
cut  a nunber  of  potentially  difficult  Issues.  In  this  section  we  will 
first  examine  the  various  methods  and  then  look  at  Issues  and  problems 
which  relate  to  them. 

Methods 

The  methods  of  collecting  diagnostic  data  fall  basically  into 
two  general  classifications:  (a)  those  which  rely  upon  the 

diagnostician's  collecting  the  perceptions  of  others,  and  (b)  those 
which  rely  upon  the  diagnostician's  own  more  or  less  direct 
perceptions.  Within  the  first  of  these  two  categories  (collecting 
the  perceptions  of  others)  two  principal  methods  have  widespread 
currency  - the  Interview  and  the  questionnaire.  These  two  are  not 
as  different  In  theory  as  they  may  appear  to  be  In  practice.  In 
the  Interview,  one  human  being  poses  to  others  In  verbal  form  a 
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series  of  questions  end  records  their  responses.  In  form,  then.  It 
relies  upon  a human  Interviewer,  and  the  questions  are  likely  to  be 
relatively  open-ended  (that  is,  calling  for  an  expressed  view  which 
Is  recorded  as  nearly  as  possible  In  the  respondent's  own  words). 
Interviews  may  be  relatively  unstructured,  in  the  sense  that  the 
questions  may  be  highly  general  ones  whose  purpose  is  to  trigger  a 
response  recorded  verbatim  In  the  respondents  awn  words,  or  a 
relatively  structured  set  of  questions  and  probes  targeted  toward 
specific  pieces  of  Information.  The  Interviewer  may.  In  fact, 
serve  the  function  In  highly  structured  Interviews  of  simply  read- 
ing a questionnaire  to  the  respondent  and  making  his  checkmarks  for 
him.  Included  within  the  general  bailiwick  of  the  Interview  for 
diagnostic  purposes  we  would  Include  group  process  meetings.  One 
ex  nple  of  this  Is  cited  by  Delbecq  and  Van  de  Ven  (1971).  In  the 
particular  Instance  which  they  describe,  a number  of  menbers  of  a 
client  system  were  brought  together  for  a meeting  to  discuss  their 
Individual  and  comnon  problems.  The  process  employed  was  highly 
structured,  with  relatively  pre-programmed  sequences  of  posting, 
digestion,  analysis,  decision  making,  and  the  like.  The  substance 
of  the  diagnosis,  however,  was  provided  by  the  participants  and 
was  generated  from  their  discussions  In  the  phased  meetings. 

Another  example,  closer  to  the  Navy's  operations,  Is  the  Coawand  Action 
Planning  Seminar  (CAPS).  Somewhat  different  from  this  might  be  the  group 
interview  where  a series  of  questions,  structured  or  unstructured,  are 
posed  by  an  Interviewer  to  a group  of  people,  and  their  collected 
responses,  or  perhaps  their  various  individual  responses,  are  noted. 
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The  paper-and-pencl 1 questionnaire  is  a second  of  the  "indirect" 
methods.  In  this  instance,  the  questions  are  highly  structured, 
specified  in  advance,  duplicated  or  printed  In  booklet  form,  and 
ordinarily  designed  to  permit  largely  closed-end  responses.  Normally, 
the  permitted  responses  take  the  form  of  multiple-choice  categories 
using  some  form  of  Likert  scale.  An  alternative  to  this  procedure, 
proposed  In  the  literature  by  Jenks  (1970),  Is  a Q-sort  in  which  a 
nunter  of  statements  are  sorted  by  the  respondent  to  an  order 
corresponding  to,  for  example,  their  description  of  him,  their 
description  of  his  supervisor,  of  the  organization,  of  some  part  of 
its  processes,  or  of  his  fellow  employees. 

The  methods  which  rely  upon  the  perceptions  of  the  data 
collector  himself  encompass  both  observation  and  records  retrieval, 
the  latter  perhaps  a marginal  member  of  this  category.  Observa- 
tion may  take  the  form  of  direct  reading,  or  of  Indirect  Inference. 
Direct  readings  may  be  person-mediated;  for  example,  an  individual 
observer  notes  the  activities,  behavior  or  reactions  of  members 
of  the  client  system  during  a particular  period  of  time.  The 
resulting  aatz  are  subsequently  used  In  diagnosis.  In  somewhat 
different  form,  observations  may  be  instrument-mediated  as,  for 
exanple,  when  audio  or  video  tapes  are  made  of  behavior  or  reaction 
segments  In  the  client  system,  and  these  tapes  are  then  submitted 
to  a diagnostic  analysis. 

In  all  of  these  direct  observation  instances,  the  purpose  Is 
to  assess  the  functioning  of  the  system  by  a procedure  which  records 
the  contents  of  that  functioning.  Somewhat  different  from  this  is 
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indirect  Inference,  also  based  upon  observation,  which  assumes  some 
of  the  characteristics  of  projective  techniques.  In  such  an  instance, 
the  observer  would  record,  not  the  substance  of  what  was  said  or  done, 
but  the  expressions  used  to  say  It  or  the  manner  In  which  it  was  done. 
He  might  then  infer  various  functional  characteristics,  not  from  the 
direct  observation  of  their  presence  or  absence,  but  from  the  words, 
terms,  and  manners  by  which  the  materia'  had  been  related.  For 
example,  non-verbal  cues,  facial  expressions,  or  posture  during  the 
Interview,  might  be  noted,  whether  the  Interviewee  attributed 
problems  to  himself  or  others  (perhaps  regardless  of  their  content), 
or  blames  factors  outside  the  system  might  be  seen  as  indicating  his 
defensiveness.  The  extent  to  which  the  Interviewee  uses  evaluative 
or  emotion- laden  terms  might  be  noted  and  seen  as  Indicative  of  one 
or  another  functional  state.  The  respondent's  degree  of  consistency 
might  be  taken  Into  account,  and  the  like.  Although  the  questions 
asked  by  the  Interviewer  might  be  identical  to  questions  asked  in  a 
direct  reading  situation,  the  material  recorded  would  be  far 
different  and  would  reflect  more  the  respondent's  manner  of  answer- 
ing than  the  substance  of  his  answer,  (c.f. , Alderfer,  1968) 

Finally,  diagnostic  material  may  be  retrieved  directly  from  the 
operating  records  of  the  organization  Itself.  Although  most  organi- 
zations do  not  maintain  updated  files  of  Information  directly 
concerned  with  the  behavior  of  members  and  organizational  processes 
such  as,  decision-making,  motivation,  and  the  like,  in  many  Instances 
material  appropriate  to  a diagnosis  of  these  aspects  of  system 
functioning  may  be  obtained  from  memoranda,  policy  statements,  and 
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the  accounting  and  control  records  maintained  by  the  organization. 
Although  the  material  entered  into  such  records  has  been,  at  one 
stage  or  another,  perceived  by  a person  other  than  the  diagnostician, 
we  class  them  here  In  the  direct  perception  category  because  they 
comprise,  in  most  Instances,  fundamental  operating  data  which  are 
then  directly  perceived  by  the  diagnostician  In  the  diagnostic  process. 

Although  a nuntoer  of  variations  on  these  methods  mty  exist--in 
fact,  the  nunfcer  may  be  inflrr  te—  there  would  appear  to  be  at  least 
reasonable  ground  for  concluding  that  they  may  be  categorized  into 
one  or  another  of  these  general  classifications.  Still,  the  goodness 
of  the  methods  Is  affected  by  a nuntoer  of  considerations  not  directly 
discernible  from  a consideration  of  the  methods  themselves,  and  it  Is 
to  these  Issues  that  our  attention  now  turns. 

Issues  and  Problems 

Cost  and  Complexity 

In  general,  observational  techniques  are  the  most  costly, 
followed  by  Interviews,  with  questionnaires  the  least  costly 
of  the  proposed  techniques  (records  retrieval  is  omitted  from 
this  conparlson  because  the  cost  issues  are  determined  In  this 
Instance  largely  by  the  Issue  of  accessibility  to  which  we 
will  turn  our  attention  shortly).  Cost  Is,  In  this  comparison, 
rather  directly  determined  by  the  amount  of  "chaff"  which  must 
be  sorted,  covered,  or  sifted  through  to  obtain  a given  amount 
of  useable,  relevant  material.  Since  observation  focuses  Its 
attention  necessarily  upon  events  as  they  occur,  all  events. 
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both  those  relevant  and  those  Irrelevant,  must  be  observed, 

although  the  latter  may  be  discarded.  The  interview,  on  the  1 

other  hand,  focuses  attention  upon  germane  issues,  at  least  to 

some  extent,  and,  hopefully  by  that  process,  eliminates  much 

of  the  extraneous  material,  recording  Instead  the  useable  and 

relevant.  It  Is  still  more  costly  than  the  questionnaire,  ‘ 

i 

however,  because  for  each  word  spoken,  another  person  must 
consume  time  in  the  listening.  The  questionnaire,  since  it 
does  not  require  a one-to-one  human  relationship  for  its 
completion,  and  since  It  prespecifies  the  material  to  be 
collected  rather  closely,  Is  certainly  the  least  costly  of 
the  three. 

Training  ano  Skill  Required 

Diagnostic  data  are  only  as  useful  as  they  are  reliable 
and  valid,  and  the  obtaining  of  reliable,  valid  data  hinges 
largely  upon  the  training  and  skill  brought  to  bear  In  the 
collection  process.  When  observational  methods  are  employed, 
the  observers  must  obviously  be  highly  skilled  and  trained. 

If  they  observe  the  functioning  situation  directly,  they  must 
know  how  to  record  their  observations,  know  the  appropriate 
amount  of  detail  to  register,  and  know  how  to  distinguish  one 
event  sequence  from  the  next,  that  Is,  how  to  know  when  one 
activity  has  stopped  and  another  has  begun.  They  must  know 
both  how  not  to  be  distracted  from  relevant  ongoing  activity 
by  peripheral  stimuli,  and  at  the  same  time,  know  which 
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peripheral  stimuli  are  In  fact  relevant  to  the  process  they  are 
supposedly  observing  and  which  they  wish  to  record.  When  the 
observation  is  instrument-mediated,  an  additional  entire  array 
of  technical  difficulties  are  encountered  which  the  subsequent 
observer-user  must  know  how  to  handle  and  solve.  Needless  to 
say,  indirect  inference— the  use  of  semi -projective  techniques— 
requires  a high  degree  of  competence  and  an  extensive  background 
In  the  projection  process  itself. 

Since  the  observer,  in  addition  to  all  of  these  difficulties, 
is  ordinarily  an  outsider,  unfamiliar  with  the  history  of  the  unit 
whose  functioning  he  Is  observing,  unaware  of  the  double,  hidden, 
and  mutually  understood  meanings  of  particular  phrases,  behaviors, 
actions,  and  cues,  it  Is  likely  that  his  readings  will  be  less 
reliable  and  less  valid  than  those  which  would  be  provided  by 
familiar  "insiders."  By  standardizing  the  stimuli  in  the  form  of 
the  questions  posed  to  the  interviewee,  and  by  relying  upon  the 
interviewee's  perceptions  and  Interpretations  of  ongoing  function- 
ing, the  face-to-face  Interview  removes,  at  least  a part,  if  not 
most,  of  the  principal  sources  of  unreliability.  To  do  this, 
however,  requires  carefully  trained  interviewers.  It  is  net 
simply  a matter  of  any  person,  with  a reasonable  degree  of 
intelligence,  traveling  through  the  organization  asking  questions 
and  noting  responses.  The  problem  of  Interviewer  bias,  as  well 
as  of  Interviewer-Induced  response  bias,  is  simply  too  great  for 
that.  Yet,  In  many  Instances,  O.D.  practioners  •’ely  upon  infor- 
mal Interviewing  as  a source  of  diagnostic  data  oblivious  to  the 
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pit-falls.  The  questionnaire,  posing  as  it  does  the  same  ques- 
tion in  the  same  form  to  all  respondents  and  relying  upon  their 
familiarity  over  a period  of  time  with  events  in  the  organization, 
goes  the  greatest  distance,  in  our  judgment,  toward  resolving  the 
problems  of  reliability  and  validity. 

Still,  what  each  of  these  methods  contributes  in  reliability 
and  validity,  it  to  some  extent  loses  in  flexibility.  Clearly, 
since  little  if  anything  is  prestructured,  observation  permits 
the  greatest  degree  of  flexibility  in  accounting  for  unique  events 
in  the  setting.  The  interview,  if  it  uses  optional  probe  ques- 
tions, may  take  at  least  some  account  of  this.  The  structured 
questionnaire  permits  little,  if  any,  of  this,  and  its  usefulness 
and  validity  in  the  larger  sense  rely  upon  the  care  and  compre- 
hensiveness which  went  Into  its  construction  at  the  outset. 
Administering  questionnaires,  of  course,  requires  some  training 
and  acquired  skill.  In  general,  however,  the  degree  of  training 
and  skill  required  for  questionnaire  administration  is  less  than 
that  required  for  interviewing  or  observation.  However,  it  should 
be  noted  that  the  amount  of  training  and  skill  going  into  question 
naire  construction  is  fully  as  great  as  the  skill  required  in  thz 
other  two  methods.  The  difference  is  that,  in  the  case  of  the 
questionnaire,  this  has  been  done  "once  for  all."  It  need  not  be 
repeated  in  each  data  collection  instance,  provided  that  a common 
or  standard  instrument  is  used. 
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The  Problem  of  N 

A diagnosis  is  as  good  as  the  data  upon  which  it  is  based. 

To  be  adequate  the  data  must  therefore  reflect  a fairly  large 
number  of  specific  instances  of  each  situation.  In  the  case  of 
the  questionnaire,  and  to  a lesser  degree  the  interview,  the 
data  collector  (the  questionnaire  itself,  or  the  interviewer) 
asks  the  respondent  to  summarize,  in  formulating  his  response, 
some  appropriate  nunber  of  occasions  in  which  a particular  type 
of  activity  has  transpired.  In  the  observational  instance, 
however,  the  nunber  of  instances  of  a particular  functional 
property  which  may  be  taken  into  account  are  those  which  have 
occurred  during  the  time-frame  of  the  observation.  This  is 
directly  a function  of  the  method  Itself,  and  means  that  a much 
longer,  and  therefore  more  costly,  period  of  information  record- 
ing must  go  on  in  order  to  encompass  the  same  nunber  of  behav- 
ioral "cases." 

The  Sampling  of  Events 

Diagnostic  data  to  be  accurate  must  constitute  a representa- 
tive sample  of  the  universe  of  behaviors  or  functional  states 
which  they  are  drawn  to  reflect.  In  the  case  of  observational 
methods,  the  sample  which  occurs  may  reflect  too  limited  a time 
period  to  make  this  possible,  or  the  existence  of  the  observer 
(the  human  being  doing  the  observing  or  the  instrument)  may  well 
itself  distort  the  events  which  it  is  intended  to  monitor.  The 
methods  which  rely  upon  the  perceptions  of  the  respondents 


22 


themselves  rely  for  the  representativeness  of  their  sampling 
upon  the  respondent's  memory  and  willingness  to  encompass  a 
sufficiently  broad  range.  In  any  specific  instance  distortions 
may  occur.  Nevertheless,  the  array  of  events  which  may  at  least 
potentially  be  taken  into  account  would  seem  to  be  larger  than 
In  the  case  of  observation.  Still  It  should  be  kept  in  mind 
that  the  demeanor  of  the  Interviewer,  or  the  wording  of  the 
questionnaire  Items,  as  well  as  the  content  encompassed  in  phras- 
ing the  Items  or  questions,  may  well  serve  to  distort  the  sample. 

Accessibility  Problems 

All  methods  suffer  to  some  extent  from  accessibility 
problems.  Not  all  participants,  nor  all  situations,  may  lend 
themselves  to  observation.  Calendars  and  time  schedules  may 
make  It  difficult  to  Interview  all  the  necessary  menters,  and 
potential  respondents  may  absent  themselves  from  questionnaire 
administration  group  sessions,  or  neglect  to  return  distributed 
or  mail-out  questionnaires.  Accessibility  becomes  the  largest 
Issue,  however,  in  relation  to  operating  records  since  in  these 
Instances,  one  Is  ordinarily  relying  for  his  information  upon 
records  and  record  keeping  systems  which  were  set  up  with  other 
purposes  than  diagnosis  In  mind.  Records  may  not  exist,  they 
may  be  tabulated  or  compiled  in  other  forms,  or  they  may  in  fact 
be  considered  confidential  and  denied  to  the  diagnostician. 
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Time  Lag  as  a Problem 

Organizations  are  dynamic  entitles,  and  events  move  across 
them  in  time,  creating  waves  or  ripple  effects  in  which  a 
series  of  events  at  one  time  in  one  part  of  the  system  cause 
other  events  at  other  points  in  time  in  the  same  or  other  parts 
of  the  system.  Thus,  there  is  an  issue  to  the  extent  that  the 
data  collection  method  used  may  not  permit  aggregation  In  the 
respondent's  mind  which  is  based  upon  his  knowing  precisely  what 
the  collected  data  are  intended  to  represent.  Today's  events 
which  are  being  observed,  for  example,  may  be  the  outcome  of 
other  events  long  since  past.  Operating  records  may  reflect 
functional  states  which  existed  several  years  previously  but 
which  no  longer  remain.  Solving  this  problem  requires  that  the 
diagnostician  not  only  know  the  nature  of  thf  constructs  which 
he  Is  measuring  and  their  place  in  an  appropriate  cause-effect 
sequence,  but  also  that  he  understand  the  relationship  between  1 

the  specific  questions  posed  or  items  sought  and  that  theoreti- 
cal framework.  Lacking  these,  he  runs  considerable  risk  of 
/ misreading  the  situation. 

Analyzing  and  Integrating  Diagnostic  Data 

Diagnostic  data,  once  collected,  are  useless  unless  submitted  to  an 
analytic,  synthetic  process  which  integrates  those  data  into  an  interpreted, 
coherent  form.  The  methods,  issues,  and  problems  of  this  aspect  of 
diagnosis  are  fully  as  important  as  the  collection  Itself. 
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Methods 

Both  quantitative,  and  non-quantltative  methods  have  currency 
in  existing  practice.  Non-quantitative  methods  include  narrative 
sunmarles  of  verbal  material  prepared  in  some  form  by  the  diagnosti- 
cian or  consultant  and  phrased  In  his  own  words  or  those  of  the 
respondents,  participant  group  derivation  sessions  in  which  the 
material  is  viewed,  reviewed,  revised,  and  analyzed  by  the  partici- 
pants and  a conclusion  or  Interpretation  generated  by  them,  and  the 
(expert)  staff  conference  In  which  a group  of  professionals,  perhaps 
differing  in  their  backgrounds,  review  the  same  or  substantive  y 
somewhat  different  portions  of  the  data  and  pool  through  a discussion 
procedure  their  conclusions  to  a general  and  meaningful  reading. 

By  nature,  these  non-quantltative  methods  are  largely  descriptive; 
although  some  use  m^y  be  made  of  quantitative  material,  the  enphasls 
Is  Instead  upon  the  narrative  description. 

Quantitative  methods,  on  the  other  hand,  require  that  the  data 
originally  collected  be  converted  to  numerical  scales,  either  through 
di'-ect  conversion  (for  example,  by  key  punching  or  by  an  optical 
scanning  scoring  process)  or  by  the  content  analysis  of  verbal  material. 
In  the  latter  case,  trained  content  analysts  read  or  listen  to  the 
original  material  and  score  each  segment  In  terms  of  preestablished 
codes . 

Issues  and  Problems 

Time-sampling,  Time-frame  Problem 

Time  enters  In  the  Interpretation,  as  It  did  in  the  collec- 
tion, as  a potentially  confounding  issue.  Great  care  must  be 
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taken  In  the  analysis  and  Interpretation  of  diagnostic  data  to 
be  certain  that  events  are  correctly  attributed  In  the  cause- 
effect  sequence  In  terms  of  the  time  sample  during  which  they 
were  collected,  or  which  was  taken  Into  account  by  the  respondent 
In  formulating  his  answers,  as  well  as  the  historical  time-frame 
which  each  functional  reading  represents.  Lacking  this,  the 
picture  which  results  may  be  a caricature  rather  than  a reasonable 
likeness,  and  the  tall  may  all  too  often  be  Interpreted  as 
wagging  the  dog.  No  hancjy  rule-of-thunt>  Is  possible  In  handling 
this  problem.  It  requires  instead  substantial  background  and 
technical  expertise  In  the  area  of  organizational  data  collection. 
Still,  In  part  the  problem  may  be  solved  by  pne-programnlng  the 
steps  and  relying  upon  conputerlzed  processes  designed  by  those 
persons  who  do  have  the  necessary  expertise. 

Expertise  Versus  Involvement 

Persons  are  known  to  be  more  motivated  by  processes  In 
which  they  are  Involved.  At  the  same  time,  diagnosis  of  an 
organization  and  its  functional  state  Is  a conplex  skill.  To 
some  extent,  therefore,  the  analytic,  interpretive  aspects  of 
diagnosis  pose  a dilemma  between  a necessity  for  bringing  to 
bear  the  expertise  of  the  technically  trained  without  sacrific- 
ing the  motivated  involvement  of  the  participants  themselves. 

It  Is  a tightrope  which  requires  a careful  tread.  Most  especially, 
It  requires  that  the  expertise  requisite  for  the  process  not  be 
delivered  In  a fashion  which  antagonizes,  becomes  overbearing, 
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or  appears  to  denigrate  the  participants,  their  knowledge,  and  < 

their  Importance. 

The  Confounding  Problem  - Horseback  Revisions 

Even  the  most  accurate  diagnosis  may  suffer  from  mid-stream 
or  horseback  revisions  made  by  the  consultant  as  he  approaches 
its  use.  Basically,  any  data  collection  and  analysis  method 
treats  with  some  degree  of  c'.re  and  accuracy  a portion,  but  not 
all,  of  the  behaviors,  events,  and  Issues  In  the  life  space  of 
the  client  system.  Some  portion  is  unique  to  that  system,  or  to 
any  group  within  it,  or  will  have  been  excluded  from  the  array 
of  Information  categories  designed  In  the  diagnostic  process  at 
Its  inception.  As  the  consultant  approaches  a particular  unit 
or  group  of  the  client  system,  he  will  necessarily  see  other 
aspects  of  what  he  feels  are  Its  functioning  not  represented  in 
the  diagnosis  which  he  has  In  hand.  Since  he  is  dealing  with  a 
real  client,  in  a real  world  situation,  the  temptation  is  well 
nigh  Irreslstable  to  revise  the  diagnosis  on  the  basis  of  his 
current  observation.  Yet,  he  Is  one  observer  observing  at  best 
a limited  and  time-bound  behavior  sample.  To  the  extent  that 
he  makes  such  revisions  he,  therefore,  very  likely  reduces  both 
the  reliability  and  the  validity  of  the  diagnosis  with  which  he 
works.  Said  otherwise,  he  approaches  each  group,  or  each  set- 
ting, as  a unique  Instance  with  live  people  and  real  problems. 

Yet  In  many  ways  the  diagnosis  and  treatment  problem  in  organi- 
zational development  Is  a "large  N"  problem.  Were  he  to  work 
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on  the  basis  of  the  diagnostic  data  provided  to  him  and  that 
alone,  given  that  it  Is  reliable  and  valid,  he  would,  across  a 
large  nunber  of  cases,  succeed  in  a high  portion  (assuming  that 
the  diagnostic  and  prescription  processes  are  themselves  high 
In  quality,  reliable,  and  valid).  Yet  he  does  not  ordinarily 
approach  his  role  with  that  degree  of  objective  detachment,  and 
each  time  that  he  yields  to  the  temptation  to  revise  on  the 
basis  of  "current  reality"  he  submits  himself  to  a situation  In 
which  his  action  steps  are  based  on  less  than  acceptably  reliable 
and  valid  data. 

Multiple  Methods  and  Redundancy 

A persuasive  case  Is  made  In  the  literature  for  the  use  of 
multiple  data  collection  and  interpretation  methods  In  the 
diagnosis.  To  the  extent  that  questionnaire  data  may  be  supple- 
mented by  interview  and  even  observational  data  and  the  same 
reality  discerned,  one  may  have  a greater  degree  of  confidence 
In  the  diagnosis  that  results.  This,  of  course,  Increases  the 
cost,  but  It  may  In  many  Instances,  If  not  most,  be  judged  worth 
the  Investment. 

Norms  and  Their  Relevance 

Diagnostic  data,  whether  quantitative  or  narrative,  are  at 
best  descriptive  until  they  are  conpared  to  a standard.  The 
standard,  of  course,  may  be  the  wording  of  the  response  cate- 
gories of  the  original  Items.  More  useful,  perhaps,  Is  the  use 
of  norms  In  which  the  behavior  or  responses,  typical  of  a group 
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or  person  like  that  being  diagnosed,  are  established  as  a 
con^arlson  standard.  Lacking  norms,  one  runs  a real  risk  of 
classifying  as  "good"  behavior  which.  In  fact,  Is  less  than 
acceptable,  and  classifying  as  "bad"  behavior  which  Is  really 
not  so.  In  all  Instances  In  which  norms  are  used,  however,  one 
should  recall  that  the  norm  Is,  at  best,  a description  of  the 
typical  behavior  of  a person  or  group  like  that  currently  being 
diagnosed;  It  does  not  necessarily  reflect  an  Ideal— nor  perhaps 
even  a desirable— state. 

Presenting  Diagnostic  Data 

Diagnostic  data  obtain  their  usefulness  when  they  are  presented  to 
persons  with  critical  roles  In  the  treatment  or  development  process. 

In  some  Instances,  diagnostic  data  are  digested  only  by  the  consultant  or 
change  agent  himself  and  represent  only  his  notes  to  himself,  perhaps  on 
tape,  perhaps  In  memo  form,  pe>  a^s  simply  retained  In  his  memory.  More 
often  than  not,  however,  presentation  of  the  diagnostic  data  Is  made 
throughout  the  client  system  with  which  subsequent  work  Is  to  be  done. 

In  such  Instances,  the  diagnostic  data  may  be  presented  In  written  form 
(that  Is,  In  the  form  of  a diagnostic  report  or  "workup"),  orally  (that 
Is,  talked  through  with  the  client  system  or  Its  key  members),  or  by  some 
conblnatlon  of  multi-media  methods,  perhaps  a narrated  report  accompanied 
by  a written  summary,  graphic  displays,  and  the  like.  A number  of  issues 
arise  In  such  Instances,  many  of  them  subject  to  substantial  disagreement 
among  practitioners. 
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Amount  of  Interpretation  Provided 

For  some  practitioners,  interpretation  provided  by  a consultant 
or  diagnostic  expert  Is  felt  best  kept  to  a minimum.  Again,  as  in 
the  analytic  process,  the  supposition  is  that  client  motivation 
toward  constructive  change  Is  greater  when  client  participation  In 
the  Interpretative  process  Is  maximal.  In  such  Instances,  diagnoses 
would  tend  to  be  presented  as  "bare  facts"  In  perhaps  tabular  or 
graphic  form  or  In  simple  anecdotal  or  descriptive  terms.  For  others, 
however,  interpretation  drawing  upon  the  best  available  skills  of 
diagnostician,  scientist,  and  practitioner.  Is  owed  to  the  client 
system. 

Complexity  Versus  Simplicity 

Whatever  the  degree  interpretation  provided,  there  would 
appear  to  be  reasonable  agreement  that  the  diagnosis  In  Its  presenta- 
tion to  the  client  system  must  be  kept  simple  enough  to  be  readily 
understood  by  its  members.  Far  from  representing  a patronizing 
stance,  this  represents  a sensible  commitment  to  minimize  or,  If 
possible  eliminate,  professional  jargon,  to  avoid  ambiguous  wording, 
and  to  make  the  Interpretive  points  which  are  presented  as  simple, 
clear,  and  straightforward  as  possible. 

Transduction 

Throughout  this  process  the  consultant  Is  a tranducer,  that  Is, 
an  Information  link  between  a body  of  knowledge  and  a user  system. 

It  is  his  task  to  convert  the  Information  which  comes  as  It  does  from 
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outside  of  the  organization's  Immediate  repertoire  to  a form  In 
which  It  can  be  freely  circulated  and  digested  Inside  the  organiza- 
tion. This  requires  a relatively  active  stance  on  the  consultant's 
part,  yet  not  one  which  Is  overbearing,  pretentious,  or  oppressive. 
Process  skills  are  required,  but  more  than  process  skill  Is  necessary. 
Substance  In  the  form  of  the  diagnosis  is  entering  the  organization 
as  a social  system,  and  to  the  consultant  falls  the  task  of  being 
certain  that  that  entering  Information  Is  put  to  constructive  uses, 
and  that  maximal  benefit  Is  gained  from  It. 

Breadth  and  Geographical  Dispersion 

When  the  organization  Is  geographically  widely  dispersed  more 
use  necessarily  must  be  made  of  written  presentation  forms.  In  such 
instances,  some  use  may  be  made  of  verbal  presentations  carefully 
spotted  in  key  locations,  but  It  Is  to  the  written  word.  Its  clarity 
and  efficiency  that  greatest  reliance  falls. 

Speed 

A diagnosis  which  consumes  months  In  the  construction  may  be 
elegant,  yet  useless,  since  much  hinges  upon  the  currency  of  the 
diagnosed  situation.  In  general,  the  faster  the  diagnosis  can  be 
returned,  the  more  relevant  and  urgent  It  will  be  to  the  client 
system  and  to  Its  efforts  to  Improve.  No  definite  turnaround  time 
guidelines  can  be  given,  and  the  time  will  necessarily  be  longer  for 
more  complex  systems,  yet  a period  of  a few  weeks  seems  the  maximum 
duration  that  may  be  safely  allowed  In  most  organizations. 
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Sunwary  of  the  Field 

In  the  pages  thus  far  we  have  reviewed  the  problem  of  diagnosis  In 
general,  the  methods  of  collecting  data,  the  methods  of  analyzing  and 
Integrating  diagnostic  data,  and  the  methods  of  presenting  the  data  so 
analyzed  and  interpreted  back  to  the  client  system.  To  reiterate  what 
has  been  stated  thus  far,  a diagnosis  requires  a comprehensive  analysis 
of  the  current  state  of  the  system,  an  analysis  which  precedes,  and  In 
part  determines,  a treatment  from  a possible  array  of  treatments.  It 
must  be  differential,  It  must  be  oriented  primarily  toward  the  client 
system's  well-being,  and  It  ought  not  be  a slnple  benchmark,  a map  of 
pitfalls  for  the  change  agent  or  consultant,  nor  a simple  earmarking  of 
the  style  differences  among  existing  consultants.  Data  may  be  collected 
by  Interview,  questionnaire,  observation,  or  from  the  operating  records 
of  the  organization  itself.  The  data,  once  collected,  may  be  analyzed 
quantitatively  or  summarized  in  non-quantltatlve  descriptive  form. 

The  picture  which  emerges  may  be  presented  in  writing,  a method  which  Is 
especially  useful  for  subsequent  retrieval  and  for  circulation  to 
geographically  dispersed  locations;  or  it  may  be  presented  orally,  or  by 
some  conblnatlon  cf  methods.  Throughout,  however,  the  diagnostician  and 
his  consultant  counterpart  have  the  responsibility  for  bringing  Into  the 
organization  and  Its  operating  situation  an  adequately  Interpreted, 
reliable,  valid,  body  of  data  which  in  relation  to  known  principles  of 
management  differentially  assess  the  current  states  of  organizational 
functioning.  It  Is  this  professional,  differential,  analytic  procedure 
which  constitutes  a genuine  diagnosis  and  which  unfortunately  Is  all  too 
seldom  provided  in  the  organizational  development  world.  In  the  present 
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instance,  we  have  undertaken  to  build  a system  to  provide  an  accurate 
differential  diagnosis  for  organizations  and  their  component  subsystems 
As  such.  It  relies  heavily  upon  the  computer,  analyses  data  from  paper- 
ond-pencll  questionnaires  In  relation  to  stored  data  from  the  operating 
records  of  organizations,  and  hopefully  provides  data  with  considerable 
speed  at  an  acceptable  level  of  accuracy  and  detail.  In  the  remainder 
of  the  report  we  will  describe  the  background,  the  general  scheme  and 
the  procedures  for  this  diagnostic  system. 


TOWARD  A COMPUTERIZED  DIAGNOSTIC  REPORT  GENERATOR 


The  computerized  diagnostic  procedure  proposed  in  the  remainder  of 
this  report  attempts  to  fullflll  the  requirements  stated  at  the  outset. 
It  is  reasonably  comprehensive,  in  that  it  assesses  the  total  functional 
state  of  the  organization  insofar  as  our  research  has  identified  the 
properties  of  that  functional  state.  Furthermore,  it  is  sufficiently 
flexible  to  permit  the  addition,  removal,  or  substitution  of  component 
characteristics  and  their  measures.  Used  with  the  Navy  Human  Resource 
Management  Survey,  with  our  own  Survey  of  Organizations,  or  with  similar 
and  derivative  instruments,  it  reflects  an  explanatory  scheme  (i.e.,  a 
set  of  management  principles)  that  has  emerged  from  years  of  rigorous 
research. 

It  assumes,  as  we  feel  a good  diagnostic  procedure  should,  that  its 
output  will  precede  and  help  determine  different  treatments  selected  to 
fit  the  states  of  functioning  that  it  describes.  Finally,  it  has  been 
constructed  with  the  thought  in  mind  that  the  purpose  in  the  end  is  the 
client  system's  (not  the  diagnostician's,  not  the  consultant’s)  greater 
well-being. 

It  further  assumes  that  data  have  been  collected  from  all,  or  at 
least  a generous  representative  sample,  of  the  members  of  the  client 
organization,  and  is  designed  with  the  paper-and-penci 1 questionnaire  in 
mind  as  its  data  source.  It  relies  extensively  upon  norms  (that  is, 
stored  measures  typical  of  various  kinds  of  client  groups). 
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The  procedure  to  be  described  draws  heavily  upon  a design  first 
formulated  In  general  terms  more  than  five  years  ago  by  menfcers  of  the 
Organizational  Development  Research  Program  staff.  The  broader  design 
was  given  by  us  the  acronym  "CANOPUS*,"  standing  for  Computerized 
Analysis  of  Organizational  Problems  In  User  Systems  and  was  viewed  as 
containing  both  computerized  diagnostic  and  conputert  zed  prescriptive 
elements  While  the  elements  were  thus  Identified  relatively  early  in 
the  history  of  the  research  Program,  the  need  was  not  recognized  as 
urgent  until  organizational  efforts  on  a massive  scale  emerged  as  a 
real  and  present  fact  In  the  Navy's  human  resource  and  manpower  programs. 
Because  diagnostic  capability  was  a clear  need  of  the  Human  Goals  effort. 
It  was  judged  most  appropriate  to  propose  that  the  diagnostic  elements 
be  developed  under  a contract  closely  tied  thereto.  The  prescriptive 
elements,  on  the  other  hand,  appeared  to  us  to  Involve  applied  research 
In  the  manpower  area,  as  well  as  development,  and  were  therefore 
proposed  for  funding  under  the  Manpower  R & D program.  Both  were  approved 
for  Initial  work.  Each  makes  greatest  sense  in  confclnatlon  with  the 
other.  Yet  It  should  be  clearly  understood  that  the  contract  under 
which  this  particular  report  Is  prepared  Is  that  which  funds  the 
diagnostic-descriptive  segments  of  the  overall  system. 


* 

Origin  of  the  acronym:  Canopus  has  multiple  referents.  In  celestial 
terms.  It  is  one  of  the  brightest  stars,  found  In  the  Corlna  (keel) 
portion  of  the  constellation  Argo  (Jason's  ship).  In  ancient  mythology, 
Argo  could  utter  prophecies  because  It  contained  a piece  of  wood  from 
the  sacred  oak  tree.  Conopus  was  also  the  pilot  of  Menelaus,  and  the 
city  In  Egypt  where  he  was  burled  assumed  his  name.  This  city  was 
famous  lor  Its  representation  of  Osiris  In  the  form  of  a vessel  or  jar, 
containing  the  vital  organs  of  the  human  system. 
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CANOPUS  - Its  Design  and  Functions 

A functional  layout  of  the  CANOPUS  Procedure,  as  it  should  ultimately 
develop,  is  presented  in  Figure  2.  As  this  chart  indicates,  the  entire 
sequence  may  be  viewed  as  containing  descriptive  functions  (those  opera- 
tions whose  purpose  Is  to  generate  suntnary  statistics  concerning  existing 
conditions)  and  analytic  functions  (those  operations  whose  purpose  is  to 
determine  the  reasons  for,  and  recomended  action  steps  to  ameliorate, 
existing  conditions).  The  components  within  these  two  broad  functions, 
plus  input,  storage,  and  text-writing  functions,  form  the  basic  steps  of 
the  procedure.  Each  is  spelled  out  in  somewhat  greater  detail  in  the 
par*gr_,phs  below: 

INPUT  - The  input  to  the  procedure  are  records  containing  the  mean 
item  and  index  scores  of  individuals  in  integral  work  groups 
In  the  organization.  (A  worn  group  is  defined  as  all 
persons  who  report  inmediately  to  a particular  supervisor.) 

CAICIJIATOR  - Survey  scores  are  relatively  useless  number  sets  until 
some  comparison  is  made  to  a standard.  Many  such  comparison 
standards  are  possible.  Scores  might,  for  example,  be 
compared  to  the  original  wording  of  the  response  alternatives. 
If  75  per  cent  of  the  members  of  a group  provide  a "5"  response 
and  this  alternative  was  worded  “very  satisfied,"  it  conveys 
at  least  limited  meaning.  However,  if  most  groups  display 
100  per  cent  In  this  "5"  category  and  the  group  at  hand  does 
not,  it  means  something  quite  different  from  what  would  be 
meant  by  comparison  to  a standard  in  which  most  groups  show 
only  25  per  cent  as  "very  satisfied."  Another  approach 


The  CANOPUS  System:  Functional  Components  and  Flow 
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would  be  to  compare  each  group  to  the  average  of  all  groups 
in  the  particular  organization  surveyed.  This  also  should 
be  dismissed  for  some  of  the  same  reasons:  no  matter  how 

excellent  or  atrocious  the  ship  or  shore  station's  record, 
half  of  the  groups  will  be  shown  as  comparatively  "good" 
and  half  comparatively  "bad."  In  the  CANOPUS  procedure  we 
have  opted  for  comparison  to  national  (e.g..  Navy-wide) 
norms  representing  a population  of  groups  like  the  one  in 
question  (same  level,  same  type  of  unit  or  function). 

The  CALCULATOR  component  takes  the  group's  mean  score  and 
converts  it  to  a percentile  score  in  the  appropriate 
normative  distribution. 

PRIORITIZER  - The  percentile  score  is  but  one  indicator  of  the 

seriousness  of  a particular  problem  or  the  contribution  of 
a particular  organizational  strength.  In  simple  terms,  it 
indicates  how  low  or  how  high  the  group  stands  in  comparison 
to  other,  similar  groups.  Lacking  any  other  information, 
both  managers  and  consultants  often  assume  that  a high  score 
(whether  In  raw  or  percentile  form)  indicates  an  area 
needing  little  or  no  attention.  Conversely,  a low  score  is, 
in  itself,  often  interpreted  as  flagging  a condition  high 
on  the  priority  list  of  matters  demanding  concern.  Yet  it 
may  not  be  so.  While,  in  general,  the  whole  array  of 
measures  relates  to  effectiveness,  for  any  particular  group 
any  single  measure  may  have  a low,  not  a high,  impact  upon 
satisfaction  and  performance.  For  this  reason,  it  may  often 
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occur  that  a measure  of  Intermediate  negativeness  is  more 
critical  In  Its  effect  than  Is  another,  "worse"  indicator. 
What  this  suggests  is  that  the  level  of  a survey  measure 
must  bs  prioritized  for  concern  and  attention  In  terms  of 
its  impact  upon  effectiveness.  PRIORITIZER  does  this  by 
weighting  each  measure  In  terms  of  its  relationship  to 
outcomes,  as  Indicated  by  standard  data  In  the  normative 
array.  The  resulting  predicted  scores  become  the  basis  for 
prioritizing  the  percentile  scores  generated  In  the 
previous  step. 

PROFILER  - From  concurrent  research  under  another  contract,  we  hate-: 
determined  that  a limited  number  of  relatively  "pure"  types 
of  groups  exist  In  the  Navy,  as  In  civilian  organizations. 
The  findings  (still  in  the  report  preparation  stage) 

Indicate  furthermore  that  these  types  respond  differentially 
to  various  action  or  development  techniques.  Both  to 
generate  additional  diagnostic  information  for  the  group 
and  its  consultant  and  to  provide  one  basis  for  treatment 
selection.  It  is  necessary  to  match  the  group  at  hand  to 
these  stored  profiles  and  Identify  the  type  with  which  It 
Is  most  consistent.  PROFILER  makes  this  match,  and  by  a 
set  of  preprogrammed  decision  rules,  identifies  the  best- 
fit  type  for  the  group  at  hand. 

D- CLASSIFIER  - Descriptive  data  must  be  aggregated  In  some  form  if 
two  requisite  conditions  are  to  be  met:  (a)  a capability 

of  grasping  their  meaning  to  the  system,  and  (b)  the 
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protection  of  confidentiality.  D-Classlfler  Initiates  this 
process  by  deciding  (according  to  pre-programmed  ruler) 
whether  the  groups  in  an  entire  le/el  (e.g.,  groups  under 
first-line  supervision)  of  an  organization  or  unit  are 
homogeneous  or  heterogeneous  in  types  Identified  by  PROFILER. 
If  the  determination  is  that  the  groups  are  homogeneous,  the 
entire  array  Is  looped  back  through  CALCULATOR  and  PRIORITIZER 
to  generate  a whole-level  reading  similar  to  that  for  any 
group.  On  the  other  hand,  if  the  determination  is  that  they 
are  different,  the  array  is  sent  to  D-DIFFERENCE  DESCRIPTOR 
prior  to  Its  resubmission. 

D-DIFFERENCE  DESCRIPTOR  - This  component  combines  sub-sets  of  the 
array  Into  tyoe-homogeneous  sets  and  submits  each  to 
CALCULATOR  and  PRIORITIZER  for  the  generation  of  whole-set 
readings  appropriate  to  the  level.  As  a result,  not  one, 
but  several  composite  readings  are  generated. 

SITUATION  ANALYZER  - Description  is  but  one  phase  of  diagnosis;  it 
provides  the  "what"  of  the  situation.  Why  conditions  occur 
Is  equally,  if  not  more.  Important.  Observable  problems 
or  deficiencies  may,  in  theory,  be  caused  by  (a)  constraining 
situations,  (b)  information  deficiencies,  (c)  skill 
deficiencies,  or  (d)  values  conflicts.  SITUATION  ANALYZER 
examines  the  extent  to  which  the  first  of  these  (constraining 
situation)  Is  associated  wi th  diagnosed  conditions.  It  does 
so  by  comparing  conditions  which  would  be  predicted  from 
our  knowledge  of  constraints  present  In  such  tnlrsgs  as  the 
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group's  organizational  climate  (and  our  knowledge  of  how 
this  typically  relates)  with  those  conditions  which  actually 
exist.  An  absence  of  an  appreciable  difference  between  the 
actual  and  situation-predicted  scores  Is  viewed  as  evidence 
for  substantial  situational  causation.  The  existence  of  an 
appreciable  difference,  on  the  other  hand.  Is  seen  as 
reducing  the  plausibility  of  situational  causation. 

PRECURSOR  - Once  the  effects  of  situational  constraints  are  elimin- 
ated, attention  necessarily  turns  to  the  other  three 
categories  (Information  deficiencies,  skill  deficiencies, 
and  values  conflicts)  as  potential  causes  of  observed 
problems.  Information  regarding  the  degree  to  which  each 
of  these  obtains  In  any  current  Instance  must  come  from 
sources  largely  external  to  the  Human  Resources  Management 
Survey.  Assuming  that  It  Is  available  and  has  been  entered, 
however,  PRECURSOR  examines  the  extent  to  which  diagnosed 
conditions  (represented  by  residual  scores  from  the 
SITUATION  ANALYZER  component)  may  be  attributed  to  a lack  of 
Information,  to  a lack  of  skill,  or  to  a conflict  In  values. 

TREATMENT  SELECTOR  - From  the  knowledge  bank  underlying  the  earlier 
PROFILER  component,  and  from  stored  history  as  It  gradually 
accumulates,  the  results  thus  far  form  the  basis  for 
selecting  one  or  more  recommended  action  steps  which  have 
the  highest  probability  of  success. 
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A-CLASSIFIER  - Although  each  group's  results  are  ultimately  printed 
out  separately  (presumably  for  the  use  by  mertoers  and  their 
supervisor),  there  remains  a necessity  of  aggregating 
causal  patterns  and  recommended  action  steps  across  whole 
levels  of  the  organization.  Both  busy  schedules  and  the 
requirements  of  action-planning  make  this  step  advisable. 
A-CLASSIFIER  performs  a function  similar  to  D-CLASSIFIER, 

In  that  It  decides  (by  pre-programmed  rules)  whether  causal 
patterns  and  action  recommendations  are  homogeneous  or 
heterogeneous  for  groups  within  the  level.  If  homogeneous, 
the  results  are  transmitted  to  storage  for  ultimate 
printout.  If  heterogeneous,  they  move  to  A- DIFFERENCE 
DESCRIPTOR  prior  to  going  to  storage. 

A-DIFFERENCE  DESCRIPTOR  - 8y  pre-programmed  rules,  this  component 
combines  sub-sets  of  groups  within  the  level  whose  causal 
and  action  statements  seem  similar.  It  then  transmits  Its 
results  to  storage. 

STORAGE  - As  Its  name  Implies,  this  component  serves  as  the  In-process 
repository  for  original  and  processed  data.  All  of  the 
necessary  results  (for  each  group  and  level)  flow  from  It  to 
the  final  component  In  the  process. 

TEXT  WRITER  - Here  are  stored  phrases,  statements,  and  paragraphs 
germane  to  each  of  the  condition-sets  which  may  result. 

From  the  Information  transmitted  to  it  from  STORAGE,  TEXT 
WRITER  prints  out  a narrative  statement  for  each  group. 
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and  for  the  system  concerning  Its  levels  and  overall 
functioning.  Included  are  prioritized  percentile  scores, 
causal  conditions,  and  recommended  action  steps. 

The  remainder  of  the  main  segment  of  this  report  will  discuss  the 
role  played  In  this  procedure  by  the  norm  matrix,  will  present  questions 
which  may  have  arisen  In  the  reader's  mind  and  hopefully  answer  them, 
and  will  end  with  a discussion  of  steps  that  remain  to  be  taken. 

The  Norm  Matrix 

As  an  earlier  section  of  the  report  stated,  any  set  of  measures,  to 
be  diagnostically  useful,  must  be  compared  to  some  standard.  A number 
are  possible;  In  this  Instance  we  have  elected  to  use  normative  data— 
i.e.,  stored  data  descriptive  of  a number  of  possible,  "typical"  client 
units— as  the  comparison  standard. 

Normative  data  are  difficult  to  obtain,  ordinarily  accumulating  In 
proportion  to  the  frequency  with  which  an  Instrument  Is  used,  the  size 
of  the  client  system  and  its  coverage  with  the  survey,  and  the  like. 

Ml  thin  any  given  body  of  normative  data,  some  trade-off  must  be  made 
between  the  usefulness  which  accrues  from  cross-cutting  It  Into  Increas- 
ingly specific  sets  (e.g.,  all  E-6's  aboard  destroyers)  and  numbers  of 
cases  (which  decline  in  each  set  as  the  nunfoer  of  sets  Increases). 

In  this  proposed  procedure  we  have  opted  for  an  Intermediate  degree 
of  stratification,  one  which  we  feel  probably  meets  the  needs  of  most 
organizational  development  practitioners  and  their  client  units.  It  Is 
shown  graphically  In  Figure  3. 
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Figure  3 


The  Norm  Matrix 
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♦Functional  types  may  be  distinguished  on  any  convenient 
basis.  Thus,  for  the  Navy,  it  might  consist  of  types 
of  ships  and  shore-stations. 
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On  the  vertical  axis  are  Indicated  rows  representing  level  of  the 
organization  at  which  any  group  might  fall.  The  basis  for  thus  providing 
different  norms  for  different  levels  is  twofold: 

(a)  Extensive  research  shows  that  conditions  simply  are  naturally 
"better"  up  the  line  than  down  below.  In  part,  this  reflects 
the  fact  that  more  capable  persons  are  selected  for  advance- 
ment In  any  organization.  In  part.  It  also  reflects  the 
greater  latitude  and  conmand  of  resources  that  exists  at  higher 
levels.  Regardless  of  the  causes,  the  condition  makes  It 
evident  that  It  Is  In  some  degree  Inaccurate  to  compare  groups 
to  a comnon  standard. 

(b)  Research  findings,  reported  elsewhere,  make  It  clear  that.  In 
the  Navy,  the  quality  of  organizational  practices  which  one 
experiences  rises  directly  with  the  reporting  level  of  one's 
group  (Franklin,  1974). 

The  horizontal  axis  indicates  functional  distinctions.  This  refers 
to  a differentiation  among  "kinds"  of  units--sh1p  types,  types  of  shore 
stations,  and  the  like.  Previous  ^search  has  Indicated  that.  In  the 
Navy,  ships  and  shore  stations  are  substantially  different  from  one 
another  and  that  ship  types  are  similarly  quite  varied.  We  anticipate 
that,  for  any  organization  like  the  Navy,  It  will  be  desirable  to  make 
the  normative  comparisons  function-specific. 

Within  any  cell  of  the  norm  matrix  are  stored  two  kinds  of  Infor- 
mation useful  to  the  diagnostic  process: 
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(a)  the  normative  values  themselves,  in  the  form  of  survey  index 
values  and  their  coordinate  percentile  equivalents; 

(b)  regression  coefficients  between  each  survey  measure  as  a 
predictor  variable  and  outcome  measures  appropriate  to  a unit 
representative  of  that  cell. 

As  an  acctmulated,  stored  history  indicates  a need  for  revision, 
both  the  percentile  values  and  the  relationship  of  each  measure  to  outcome 
variables  (e.g.,  satisfaction,  retention,  health,  or  operating  performance) 
may  be  modified  to  make  them  more  current.  Furthermore,  since  the 
procedure  relies  upon  stored  values  typical  of  the  cell,  any  values  may 
be  inserted,  thus  providing  the  potential  for  simulation  studies  to  aid 
personnel  policy-makers. 

The  jud^nent  concerning  how  frequently  to  revise  and  update  norms 
is  arbitrary  and  the  criteria  necessarily  somewhat  vague.  It  must  be 
sufficiently  frequent  to  keep  the  normative  array  current,  yet  not  so 
frequent  as  to  disrupt  the  ability  to  make  progress  comparisons.  Of 
course,  as  the  array  becomes  larger,  the  addition  of  any  new  increment 
has  less  impact  upon  the  stored  values.  As  an  arbitrary  rule,  perhaps  a 
frequency  of  once  per  year  might  be  considered  an  optimal  inuarval  at 
the  outset. 

Issues  and  Answers 

Finally,  there  are  certain  issues  and  questions  which  doubtless 
have  occurred  to  the  reader  which  ought  be  addressed.  They  are  posed 
below  in  the  form  of  questions,  followed  by  brief  responses. 
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What  me ob ure 8 form  the  basis  for  the  diagnosis?  < I 

Although  any  measures  having  the  same  format  could  be  used,  | 

In  the  present  Instance  the  procedure  uses  the  Items  and 
indices  from  the  Navy  Human  Resource  Management  Survey 
(or  Its  civilian  antecedent,  the  ISR  Survey  of  Organizations). 

Thus  the  measures  tap  organizational  (command)  climate, 
supervisory  leadership  behavior,  subordinate  peer  behavior, 
emergent  group  processes,  and  outcome  measures,  as  well  as 
special  purpose  measures  In  a nurber  of  areas. 

What  ie  the  analysis  unit  for  diagnostic  purposes? 

The  face-to-face  work  group  (defined  as  the  lnwedlate 
subordinates  of  a supervisor)  Is  the  analytic  unit,  since 
a considerable  amount  of  research  has  Indicated  that  groups, 
rather  than  individuals,  are  the  basic  building  blocks  of 
organizations.  In  this  vein,  the  procedure  uses  the  group 
mean  score  as  Indicative  of  the  perception  of  the  "averageu 
or  "typical"  group  member,  and  it  Is  this  measure  for  which 
a percentile  score  Is  generated. 

WJiat  cauee-and-effect  assumptions  are  made  in  this  procedure? 

It  assumes  that  the  organization  Is  an  open  social  system. 

In  which  nothing  really  "exists"  but  the  behavior  of  its 
menters.  These  behaviors  combine  to  form  characteristics 
(called  "emergent  properties")  that  are  different  from 
(or  perhaps  greater  or  less  than)  the  sum  of  the  component 
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behaviors.  Thus,  group  processes  are  seen  as  characteristics, 
above  and  beyond  the  behaviors  )f  group  members,  which  emerge 
from  those  behaviors.  Organizational  climate  is  similarly  an 
emergent  characteristic,  resulting  from  the  group  processes 
of  numerous  groups  and  affecting  (as  a situational  constraint) 
the  functioning  of  some  other  group.  Perhaps  the  most  familiar 
emergent  property  of  any  group  is  its  performance.  Evidence 
reinforcing  the  validity  of  this  sequence  in  both  civilian  and 
Navy  settings  has  been  provided  in  several  recent  technical 
reports  (Franklin,  1973,  1974).  Behaviors,  themselves,  are 
seen  as  caused  by  any  of  four  kinds  of  factors:  information, 

skill,  values,  and  the  situation.  These  antecedent  character- 
istics are  termed  ‘'precursors"  and  a comprehension  of  them  is 
seen  as  adding  to  the  diagnostician's  ability  to  understand 
why  behavior  of  a particular  form  occurs  (Bowers,  Franklin 
and  Pecorella,  1973). 

Are  the  measures  stored  in  the  nom  matrix  taken  from  thie  group 

iteelf,  or  from  gome  other  source? 

By  definition,  normative  data  are  those  presumed  to  be 
representative  of  a whole  population  or  sub-population. 
Therefore,  each  cell  of  the  norm  matrix  contains,  at  any 
given  time,  all  of  the  data  thus  far  collected  from  persons 
in  groups  identified  as  belonging  to  the  cell.  Unless 
previous  waves  of  collected  data  have  been  entered  into  the 
matrix  and  the  present  group  was  previously  surveyed,  none 
of  the  data  in  the  cell  will  have  come  from  the  group  at  hand. 
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Similarly,  the  relationships  to  outcomes  (regression  coef- 
ficients) stored  in  the  cells  of  the  matrix  are  those  found 
to  be  typical  for  groups  of  this  kind.  Of  course,  in  using 
the  procedure,  a large  organization  will  want  to  accumulate 
its  own  values  and  coefficients  from  large  numbers  of  its 
own  groups.  These  values  and  coefficients  will  then  serve 
the  necessary  normative  functions. 

Why  must  no  me  be  external  to  the  unit  being  diagnosed? 

As  the  text  indicates,  "internal"  comparisons--! .e. , 
comparisons  of  each  group  to  the  average  of  groups  In  its 
own  unit--forces  half  to  be  "bad,"  half  to  be  "good,"  and 
can  be  quite  misleading.  Of  course,  the  same  situation 
occurs  In  theory  when  comparison  Is  made  to  national  norms: 
half  of  the  national  arri^y  is  portrayed  as  good,  half  bad. 

Yet  the  difficulty  declines  in  direct  proportion  to  the 
breadth  of  the  array.  A group,  aboard  a destroyer,  which  is 
compared  to  the  average  group  aboard  that  destroyer  only,  runs 
maximum  risk  of  comparative  distortion.  When  compared  to  alJL 
such  groups  aboard  all  destroyers,  the  distortion  is  vastly 
reduced. 

What  Mas  Been  Accomplished  and  What  Remains  To  Be  Done 

Of  the  components  viewed  at  the  outset  as  suitable  for  imnediate 
work,  two  are  presented  In  the  appendices  to  this  report  as  substantially 
complete  (CALCULATOR  and  PRIORITIZER).  Appendix  A presents  a general 
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description  of  the  software  contained  In  these  two  components,  together 
with  a copy  of  the  computer  program  itself.  Appendix  B presents  sample 
output  from  the  two  components. 

Of  the  components  which  remain,  some  are  viewed  as  relatively  simple 
and  straightforward,  whereas  others  {such  as  SITUATION  ANALYZER)  are 
deemed  more  complex.  In  the  months  which  remain,  our  effort  will  be 
directed  toward  these  remaining  steps,  especially  toward  the  more 
complex  ones. 
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Appendix  A 


Description  and  Computer  Program  for 
the  Calculator  and  Prioritizer  Functions 


PNORM 

GENERAL  DESCRIPTION 

PNORM  performs  two  basic  functions  utilizing  data  from  two  data  sets, 
identified  as  MASTER  NORMS  and  SITE  NORMS.  These  functions  are: 

A.  Percentile  score  construction  and  scanning  for  the  SITE  NORMS 
data  based  on  (Al)  MASTER  NORMS  deciles  (A2)  deciles  computed 
from  the  SITE  NORMS  data  set  alone  or  (A3)  deciles  computed 
using  a data  set  at  some  prior  time. 

B.  Ranked  predicted  criterion  score  construction  and  scanning. 
Predicted  criterion  scores  are  constructed  by  a simple 
regression  model  relating  functional  measures  to  criterion, 
either  usiag  (Bl)  MASTER  NORMS  to  establish  regression 
coeffi cients  and  deciles  for  application  to  SITE  NORMS  or 
(B2)  using  SITE  NORMS  alone,  or  (B3)  applying  a set  of 
regression  coefficients  and  deciles  computed  using  a data 
set  at  some  prior  time. 

FUNCTION  A.  There  are  three  "modes"  in  which  function  A.  can  be  executed, 
Al , A2  and  A3.  Al  assumes  that  a MASTER  NORMS  data  set  is  used  currently 
to  establish  deciles  for  the  SITE  NORMS  data,  A2  assumes  the  SITE  NORMS 
will  be  used  to  generate  its  own  decile  sets  and  A3  assumes  that  deciles 
have  been  confuted  by  a prior  computer  run. 

MODE  Al.  Here  the  MASTER  NORMS  data  set  is  read  and  aggregated  into 

groups  ordered  by  stratum  (see  keywords  (GVAR,  SVAR)  and  deciles 
computed  in  STRATUM/VARIABLE  order.  They  are  written  using 
unformatted  FORTRAN  on  the  permanent  disk  file  FT04F001 . Next, 
the  SITE  NORMS  data  set  is  read,  aggregated  and  sorted  into  *he 
same  STRATUM/GROUP  set  and  the  MASTER  NORMS  deciles  used  t 
convert  the  aggregated  data  into  percentile  scores.  These 
percentile  scores  are  then  scanned  for  "outlying"  variables 
within  groups  defined  by  keywords  PMIN,  PMAX  and  the  outliers 
are  printed  for  eac'  group.  Keywords  MASTER,  NORM  and  INFI 
are  used  in  this  moae. 

MODE  A2.  This  Is  the  same  as  Al  except  that  the  SITE  NORMS  data  set  is 
used  to  compute  its  own  deciles,  rather  than  using  those  from 
a MASTER  NORMS  data  set.  The  aggregation,  sorting,  per- 
centlzlng  and  scanning  remains  the  same.  Keywords  NOMA  and 
INFI  are  relevant  here. 
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MODE  A3.  Here  the  procedure  is  the  sane  as  In  A1  except  that  the  MASTER 
NORMS  data  set  has  been  used  at  some  prior  point  in  time  and 
only  the  decile  sets  (residing  in  permanent  disk  FY04F0Q1)  are 
used.  Again  the  aggregation,  sorting,  percentizing  and 
scanning  is  the  same.  Keywords  NORM  and  INFI  are  relevant. 

FUNCTION  B.  This  function  can  again  be  implemented  through  three  alterna- 
tive modes , depending  on  whether  the  MASTER  NORMS  data  set  is  used 
currently,  or  was  used  in  a prior  computer  job,  to  estimate  the  regression 
coefficients. 

MODE  B1 . In  the  first  mode,  the  MASTER  NORMS  data  set  is  used  to  generate 
a set  of  simple  regression  coefficients  between  the  functional 
measures  and  a criterion  variable.  Since  all  variables  must  be 
in  standard  score  form  (i.e.,  means  of  zero  and  standard 
deviation  of  one)  in  all  three  modes,  these  "Beta"  coefficients 
are  Identical  to  the  Pearson's  r's  between  the  same  pairs  of 
variables.  If  the  data  are  not  in  standard  score  form,  the 
keyword  STAN  must  be  used  to  standardize  the  variables  internal 
to  the  PNORM  program  prior  to  the  regression  calculations. 

The  obtained  regression  coefficients  are  then  applied  to  the 
SITE  NORMS  data  in  a second  job  step  to  produce  a set  of 
PREDICTED  CRITERION  SCORES.  These  PREDICTED  CRITERION  SCORES 
are  then  scanned  for  outliers  in  the  same  manner  as  in  function  / 
above.  MODE  1 is  obtained  by  Indicating  the  keywords  PREDICTION 
and  MASTER  on  the  parameter  card. 

MODE  B2.  In  the  second  mode  everything  is  as  above,  except  that  only  the 
SITE  NORMS  data  set  is  used.  First,  simple  regression 
coefficients  are  computed,  again  using  the  functional  measures 
as  predictors  and  the  criteria  as  dependent  variables.  Now, 
however,  the  predicted  criterion  scores  are  arrived  at  using 
the  same  SITE  NORMS  functional  measures  as  predictors.  Keywords 
PRED  and  NOMA  are  invoked  to  implement  this  mode. 

MODE  B3.  The  third  mode  assumes  that  regression  coefficients  and  deciles 
have  been  calculated  by  a prior  procedure  using  a relevant 
data  set  and  have  been  placed  In  the  permanent  disk  data  set 
defined  by  DSRN  IN.  These  coefficients  are  read  by  the  program 
and  then  applied  against  functional  measures  from  the  SITE  NORMS 
data  set  to  generate  predicted  criterion  scores.  Keywords  PRED 
and  REGR  are  used  for  this  mode. 

Input  Input  to  the  program  consists  of  an  OSIRIS  data  set  and  control 
cards. 

Output  The  output  consists  of  the  program  printout,  and  sets  of  simple 
regression  coefficients  in  DSRN  IN  (if  the  PREDICTED  CRITERION 
SCORE  mode  is  chosen). 
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STANDARD  OSIRIS  FEATURES 


Case  and  variable  selection.  The  OSIRIS  standard  global  filter  option  Is 
available  to  select  a subset  of  cases  from  the  Input  data.  In  addition, 
the  keyword  parameter  BADDATA  allows  the  user  to  skip  cases  having  "BAD" 
values  (e.g. , blanks  or  special  characters). 

Transforming  data.  OSIRIS  Recode  statements  In  decimal  mode  may  be  used 
wl th  the  program.  (See  ref.  1 In  Appendix  B of  the  OSIRIS  Manual) 

Missing  data.  The  keyword  parameter  MDATA*(NONE,  MD1,  MD2,  BOTH)  allows 
the  user  to  exclude  either  MD1,  MD2  or  both  "missing  data"  values. 

(See  global  parameters) 


MttNTED  OUTPUT 


The  major  conponents  of  the  printout  are  specified  below. 


For  Function  A: 

A.  Interpretation  of  control  cards  and  listing  of  Input  dictionary. 

All  options  are  given  program  Interpretation  and  the  input  dlctl onary 
(comprised  of  the  GROUP  variable,  STRATA  variable,  and  all  NORM 
variables)  is  printed. 


B.  Aggregated  Data  Matrix.  A matrix  whose  rows  are  groups  and  columns 
norm  variables  Is  printed.  The  elements  represent  mean  values  for 
all  cases  In  the  data  set  having  the  particular  group. 

C.  Peel le  Printout.  (Optional)  A matrix  of  data  for  each  strata  Is 
printed  giving  the  deciles  for  each  variable  based  on  the  mean 
values  for  all  groups  within  the  particular  strata. 

D.  Percentlzed  Matrix.  A matrix  of  data  corresponding  to  the  aggregated 
data  matrix  whose  values  are  percentile  scores  associated  with  the 
aggregated  data  matrix. 

E.  Norm  Extremes.  A listing  of  extreme  NORM  or  CRITERION  scores  for  each 
group  determined  by  the  MAXP,  MINP  keyword  parameters.  The  output  has 
the  following  format: 

***  LISTING  OF  NORM  EXTREMES  BY  GROUP 
RANK  ORDERING  BY  PERCENTILE  MAGNITUDE 


One  set 
for 
each 
group 


STRATA  NO.  xxx  GROUP  NO.  xxx 
HIGH  NORMS 

VARIABLE  NUK1ER 

> 

LOW  NORMS 

VARIABLE  NUMBER 


SAMPLE  SIZE  * xxxx 
PERCENTILE 

f 

PERCENTILE 
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etc.,  where  the  variable  nunfcet  within  the  HIGH  NORM  and  LOW  NORM  sets 
are  printed  by  ascending  percentile  order. 

All  the  previous  printout  Is  listed  once  for  the  MASTER  NORMS  data  set 
and  again  for  the  SITE  NORMS  data  set,  subject  to  the  DECILE/NODECILE 
keyword,  allowing  decile  printout  expression  on  either  or  both  data  sets. 


For  Function  B: 

Ail  of  the  above  A-E  components  are  printed.  However,  the  DECILE 
printout  gives  the  deciles  for  each  group  across  all  variables  rather  than 
by  variable  across  all  groups  In  a strata  as  In  Function  A.  The  PERCENTIZED 
MATRIX,  It  Is  reminded,  has  been  confuted  using  the  ordering  of  the 
predicted  criterion  scores  for  each  group.  In  addition,  the  following  two 
matrices  are  printed  (a. ter  B.  - Aggregated  Data  Matrix). 

1.  Standardized  Data  Matrix.  The  matrix  of  aggregated  data 

transformed  (by  variable  within  strata)  to  standard  scores,  l.e., 
STD.  SCORE- (SCORE-MEAN) /STD.  DEV. 

II.  Predicted  Criterion  Matrix.  Matrix  giving  the  results  of  applying 
tlie  single  regression  coefficients  to  the  standardized  data  (B1 . ) . 
The  percentlzed  predicted  criterion  score  matrix  Is  scanned  in  the 
same  manner  and  has  the  same  output  format  as  described  for 
Function  A.  (printout  E.)  above. 


INPUT  OSIRIS  DATA  SET 


Data  must  be  Input  In  the  form  of  an  OSIRIS  data  set,  type  1 or  type  3. 

A maximum  of  202  variables,  Including  the  group  variable,  strata  variable 
and  all  norm  variables  may  be  used  In  a run.  A maxliman  of  220  variables 
total  (l.e.,  Including  recode  type  variables)  may  be  used. 


RESTRICTIONS 

1.  Maximum  nunfcer  of  groups  Is  300. 

2.  Maximum  nunfcer  of  strata  Is  100. 

3.  Maximum  nunfcer  of  NORM  variables  Is  200. 

4.  Maximum  total  nunfcer  of  variables  Is  220.  (Includes  group,  strata, 
norm  and  recode  variables.) 

5.  Each  group  must  have  a unique  stratum  associated  with  it. 
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6.  The  strata  variable  for  the  SITE  NORMS  data  set  must  have  codes  that 
match  the  MASTER  NORMS  codes  exactly.  (The  1st  stratum  code  decile 
set  Is  used  to  percentlze  the  aggregated  data  for  the  1st  SITE  NORMS 
data  set  stratum,  the  2nd  stratum  code  decile  set  Is  used  to 
percentlze  the  aggregated  data  for  the  2nd  SITE  NORMS  data  set 
stratum,. . . , etc. ) 

7.  No  limit  on  nunber  of  cases. 

8.  Maximum  nunber  of  groups  within  one  stratum  is  100. 


TEMPORARY  DISK  STORAGE 


There  are  three  temporary  disk  data  sets  used  by  the  PNORM  program.  They 
are  referred  to  symbolically  as  ITEH>,  ISAVE  and  IN  in  SUBROUTINE  INPUT 
and  are  presently  assigned  DSNR's  3,  4 and  7,  respectively.  The  functions 
of  these  data  sets  is  as  follows: 

ITEMP  Is  used  In  SUBROUTINE  SORT  to  hold  a complete  copy  of  the  fullword 
input  data  and  uses  unformated  FORTRAN  Read/Write  statements.  Thus,  the 
amount  of  space  required  is  N*NV*4  bytes,  where  N is  the  number  of  cases 
passing  the  filter,  missing  data  and  bad  data  checks  and  NV  Is  the  number 
NORM  variables  plus  three. 

ISAVE  is  used  to  store  the  deciles  generated  by  the  MASTER  data  set  job 
step.  Writing  is  done  In  SUBROUTINE  PCENT  and  the  deciles  are  read  back 
in  and  used  for  percent! zing  the  SITE  NORMS  data  In  SUBROUTINE  MACHO. 

Space  required  Is  NV*NS*44  bytes  where  NS  Is  the  number  of  strata. 

IN  is  used  to  read  regression  coefficients  set  up  prtor  to  the  running  of 
a PREDICTED  CRITERION  SCORE  job  step.  Since  there  must  be  one  regression 
coefficient  for  each  criterion  variable  for  each  strata,  the  space 
requirement  is  NV*NS*4  bytes. 

Both  ISAVE  and  IN  are  also  used  with  unformated  FORTRAN  READ/WRITE. 


EXECUTING  THE  PROGRAM 


The  Job  Control  Language,  monitor  control  cards  and  program  control  cards 
needed  to  execute  PNORM  are  outlined  below.  Cards  must  be  supplied  In 
the  indicated  order.  Refer  to  Appendix  A of  the  OSIR's  Manual  for  details 
on  the  OSIRIS  Monitor  and  Its  catalogued  procedure  and  Appendix  E of  the 
OSIRIS  Manual  for  assistance  with  JCL.  The  'xxxx'  in  the  ddnames  that 
follow  are  determined  by  the  parameter  INFILE. 
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//  EXEC  OSIRIS 

//DICTxxxx  OD  Describe  the  input  dictionary 

(Omit  this  OD  card  if  $DICT  is  used.) 
//DATA xxxx  DD  Describe  the  input  data  file. 

(Omit  this  DD  card  if  $DATA  is  used.) 

//SETUP  OD  * 

$RUN  PNORM 

SRECODE  (Optional) 

Recode  statements 

SSETUP 

1.  Global  filter.  (Optional) 

2.  Label  card. 

3.  Global  parameters. 

4.  Variable  list. 

$DICT  (Optional) 

Dictionary  cards. 

$DATA. (Optional ) 

Data  cards. 


/* 


PROGRAM  CONTROL  CARDS 


Refer  to  Appendix  C of  the  OSIRIS  Manual  for  detailed  descriptions  of  the 

standard  OSIRIS  program  control  cards.  Items  1-4  below. 

1.  Global  filter.  (Optional)  Selects  a subset  of  cases  to  be  used 
fn  the  runs. 

Example:  INCLUDE  V25-2-9* 

2.  Label  card.  One  card  containing  up  to  80  characters  to  label  the 
printed  output. 

Example:  PNORM  RUN  FOR  MALES  ONLY 

3.  Global  parameters.  Parameters  are  chosen  from  thosedescribed  below, 
must  be  separated  by  blanks  and/or  cownas,  and  must  be  terminated 
with  an  asterisk.  Defaults  are  underlined. 

Example:  GVAR-1 , SVAR-2  DECILES* 

SVAR- variable  number  The  variable  nunfcer  to  be  used  as  the 

stratum  variable. 

GVAR*> variable  nunfeer  The  variable  nunber  to  be  used  as  the 

group  variable. 
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PMAXa60/I 

PMIN-40/I 

HOSORT/SORT 


lNFI«IN/xxxy 

NODE/DEC1 


The  maximum  percentile  value  used  for 
scanning  of  the  percentized  matrix. 

The  minimum  percentile  value  used  for 
scanning  of  the  percentized  matrix. 

Whether  or  not  the  Input  data  has  to 
be  sorted.  (If  sorted  the  data  must 
be  ordered  first  by  the  SVAR  variable, 
then  by  the  6VAR  variable. 

Input  data  ddname  suffix. 

Whether  or  not  deciles  for  each 
variable  within  each  stratum  should 
be  computed  and  printed. 


BADDATA-STOP/SKIP/MDI /MD2 

When  non-numeric  characters  (including 
imbedded  blanks,  &'s,  and  -'s  and 
all-blank  fields)  are  found  in  numeric 
variables,  the  program  should: 

STOP:  Terminate  the  run. 

SKIP:  Skip  the  case. 

MD1:  Recode  a full  field  of  & to  a 

full  field  of  nines  plus  1 
(i.e.,  recode  & to  10,  &&  to 
100,  etc.).  Recode  a full  field 
of  - to  a full  field  of  nines 
plus  2 (I.e.,  recode  1 to  11, 

--  to  101,  etc.).  Recode  all 
other  non-numeric  values  to  the 
first  missing  data  code. 

MD2:  Recode  full  fields  uf  & and  - as 

specified  in  MD1  above.  Recode 
all  other  non-numeric  values  to 
the  second  missing  data  code. 

For  SKIP,  MD1 , and  MD2  a message  Is 
printed  about  the  number  of  cases  so 
treated. 

MDATA*B0TH/MD1MD2/N0NE  Eliminate  cases  from  the  analysis  that 

have  MD1 , MD2,  either  (BOTH)  missing- 
data  values  on  an^  variables  from  the 
analysis.  If  mTssIng-data  Is  to  be 
Included  NONE  should  be  chosen. 


NOPRED/PRED 


Whether  the  RANKED  PREDICTED  CRITERION 
score  mode  Is  chosen. 
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MAST/NQMA/NQRM  Indicates  the  Input  mode  for  the 

PERCENTILE  SCORE  CONSTRUCTION/SCANNING 
option. 

MAST  Indicates  that  the  Input  data  used 
In  this  Job  step  Is  the  MASTER  NORMS 
data  and  that  the  following  job  step 
will  use  a NEW  NORMS  data  set  as  Input,' 
have  the  keyword  NORM  and  be  percentlzed 
according  to  the  deciles  established 
using  the  MASTER  NORMS. 

NOMA  means  that  the  current  job  step 
has  data  that  will  be  percentlzed 
according  to  Its  own  deciles  and  not 
used  to  establlshUecIles  for  data 
from  any  subsequent  job  step. 

NORM  Indicates  that  a MASTER  NORMS 
data  set  has  already  been  used  to 
generate  deciles  to  be  used  for  this 
step.  The  deciles  are  assumed  to  be 
stored  on  DSRN  defined  by  ISAVE. 

NORE/REGR  For  the  PREDICTED  CRITERION  SCORE 

(PRED  keyword)  mode  only.  REGR  Indicates 
that  the  simple  regression  coefficients 
have  been  calculated  In  a prior  run  and 
are  stored  In  DSRN  IN,  arranged  In 
strata/ variable  list  order.  NORE,  the 
default.  Indicates  that  the  simple 
regression  coefficients  are  to  be 
calculated. 

NOST/STAH  For  the  PREDICTED  CRITERION  SCORE 

(PRED  keyword)  mode  only.  The  keyword 
STAN  Indicates  that  the  data  will  be 
standardized  by  the  program  before  the 
regression  coefficients  are  computed. 

NOST  Indicates  that  they  will  not  be 
standardized  by  the  program. 


As  an  example  of  the  use  of  these  keywords,  consider  the  following 
three  job  steps: 


//  EXEC  OSIRIS 
//OICTIN  DD  DSN-MASTERDI 


//DATAIN  DO 
//D1CTINA  DD 
//DATAINA  DD 
//DICTINB  DD 


DSN-MASTERDA 

DSN-NEWDI 

DSN-NEWDA 

DSN-DICT 


A 
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//DATAINB  OD  DSN = DATA 
//SETUP  DD  * 

$RUN  PNORM 
$RECOOE 

^ (Recode  Statements) 

SSETUP 

INCLUDE  Vl=l  * 

(1)  MASTER  NORMS  DATA  (MALES  ONLY) 

SVAR=2  GVARS  3 * 

* V4-V10  * 

SEND 

SRUN  PNORM 
INCLUDE  V1*1  * 

(2)  NEW  NORMS  DATA  (MALES  ONLY) 

SVAR=2  GVAR=3  NORM  INF1=A  * 

V4-V10  * 

SEND 

SRUN  PNORM 
EXCLUDE  V20*2  * 

(3)  PERCENTIZING  RUN  ON  NON-BLACKS  (V 

SVAR=1  GVAR=2  NOMA  INFI=B  * 

V3-V10  * 

/* 

In  job  step  (1)  the  data  sets  MASTERDI  and  MASTERDA  are  used  as 
MASTER  NORMS  data.  After  being  filtered  to  include  males  only 
(INCLUDE  Vl  = l *),  they  are  used  to  construct  deciles  for  use  in  job 
step  (2).  Here  the  data  sets  NEWDI  and  NEWDA,  assigned  through  the 

* keyword  INF1=A,  are  used  as  the  NEW  NORMS  dictionary  and  data  and 

percentized  according  to  deciles  established  in  job  step  (1).* 

In  job  step  (3),  the  data  sets  DICT  and  DATA,  assigned  through  the 
keyword  INFI=B,  are  used  to  generate  a percentized  matrix,  for  the 
/ same  data,  as  final  output.  A filter  to  include  non-blacks 

(EXCLUDE  V20»2*),  is  also  used. 

4.  Variable  List.  Contains  all  norm  variables.  Also  If  the  predicted 
criterion  function  is  used  and  the  SITE  NORMS  data  is  not  being 
used,  then  the  last  variable  indicated  is  used  as  the  criterion 
variable  in  regression  coeff.  calculations. 


* 

It  should  be  remarked  that  there  is  a small  amount  of  linear  Interpolation 
error  associated  with  this  procedure.  Tests  using  the  same  data,  a 
MASTER  NORMS  data  set  with  480  cases  and  a SITE  NORMS  data  set  of  125 
cases  on  3 NORM  variables  indicate  errors  of  less  than  .01-. 02  for  the 
percentized  matrix.  An  accuracy  improvement  for  this  procedure  would  be 
to  use  percentiles  Instead  of  deciles.  Because  of  the  additional  CPU 
involved  and  the  marginal  accuracy  improvement  potential,  deciles  are 
presently  used. 
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Sample  Printouts  of  the 
Calculator  and  Prioritizer  Functions 
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